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DETAILED ACTION 
Response to Amendment 

1 . This office action is in response to comnnunications filed 03/24/2008. Claims 1 
and 35 are amended. Claims 2-9, 19, 20, 36-43, 48, 49 are original. Claim 14, 17, 18, 
21, 46, 47, 50-52, 55, 56 and 61-72 are previously presented. Claims 10-13, 15, 16, 22- 
34, 44, 45, 53-55, 57-60 are cancelled. 

Response to Arguments 

2. Applicant's arguments filed 03/24/2008 have been fully considered but they are 
not persuasive. 

Regarding Claims 1 and 35, the applicant argues that Wistendahl does not 
disclose the claimed object mapping table that stores an identifier to such N data (Page 
15). The applicant further argues that Wistendahl fails to teach separate mask data 
(Page 15). The applicant argues that user of Wistendahl's system selects a particular 
image, an IDM program is invoked to compare the selected coordinates with the hotspot 
coordinates stored as N data to determine whether the selected image is indeed a 
hotspot and there is not identifier (Page 15). The applicant argues that Wistendahl's N 
data simply defines a hot spot is akin to coordinate tracking data (Page 15). The 
applicant also argues that Srinivasan's invention can do everything Wistendahl's 
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invention can do if not more (Pages 15-16). Tlierefore, even N data may constitute a 
data structure it is not an identifier to an information data structure (Page 16). Tlie 
applicant argues tliat tlie tliere is no suggestion or motivation to modify Wistendalil's N 
data were to identify tlie IDM program, it would not identify one of a plurality IDM 
programs. The applicant also argues that there is no motivation to combine (). 

In response to the argument, Wistendahl discloses an object mapping table or N 
data information which includes N data indexed to specific type of interactive use of the 
media content (Column 7, lines 55-57, Column 6, lines 17-59, Figure 2). Microsoft 
Press 3'"'^ edition Computer Dictionary defines data structure as an organization 
scheme, such as a record or array that can be applied to data to facilitate interpreting 
the data or performing operations on it. The N data is an organization scheme which 
applied to data to facilitate interpretation of the data or performing operations or for the 
interactive use of the media content. 

Wistendahl discloses the mask of objects including an identifier or an object 
identifier for A'(F) identifying object A or hotspot B'(F) identifying object B (Column 6, 
lines 17-38, Column 7, lines 1-10, Figure 2, Figure 3). Wistendahl discloses a mask 
including an identifier to the object mapping table and the object mapping table having 
indicia from the corresponding mask to storing an identifier to a corresponding one of a 
pluriaty of information data structures including A'(Fi) or B'(Fi) (Column 6, lines 17-38, 
Column 10, lines 8-67) and the corresponding information data structure including 
information associated with the particular video object (Column 10, lines 57-67, Column 
11, lines 1-40, Column 12, lines 1-18, 40-49). Srinivasan discloses identifying the 
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corresponding nnasl< if tlie comparison results in a matcli (Column 7, lines 36-45, 
Column 12, lines 21-46, Column 21, lines 34-52). Wistendahl discloses the identifier. 
In this case, the motivation to combine is to provide media content without locking the 
content to a particular platform and to reduce the development time (Column 2, lines 35- 
45). These arguments are very similar to those made in previous remarks by the 
applicant. The Examiner respectfully disagrees with the applicant as the inventions of 
Srinivasan and Wistendahl meet the limitations for the applicant's claimed invention. 

Furthermore, in KSR International Co. Teleflex Inc., 127 S.Ct 1727, No. 04-1350, 
slip. op. at 12 (2007), the Court found that if all the claimed elements are known in the 
prior art then one skilled in the art could have combined the elements as claimed by 
known methods with no change in their respective functions, and the combination would 
have yield predictable results to one of ordinary skill in the art at the time of the 
invention. 

3. Regarding Claims 2-9, 14,17-21, 36-43, 46-52, 56, 61-62, 64-66 and 68-70 and 
70, the applicant argues that Srinivasan and Wistendahl cannot meet the limitations 
because they depend on allowable base claim (Page 16). 

In response to the argument, see response to claims 1 and 35. 
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4. Regarding Claims 56, 66 and 70, tlie applicant argues that Wistendahl cannot 
meet the limitations because Wistendahl does not disclose the claimed object mapping 
table (Pages 16-17). 

In response to the argument, Wistendahl does disclose the object mapping table. 
See response to claims 1 and 35. Therefore, Wistendahl meets the limitations of 
Claims 56, 66 and 70. 

5. Regarding Claim 72, the applicant argues that Kaiser does not disclose the 
visibility bit which will identify an object via highlighting (Page 17). 

In response to the arguments. Kaiser discloses a digital system which includes 
programmable logic to perform functions (Column 7, lines 35-37). Kaiser discloses the 
digital system includes indicators designating a visual highlight (Column 10, lines 20-38, 
Column 7, lines 38-55). The system necessarily includes bits that enable processing of 
data. Therefore, the visual highlights of the reference disclose a visibility bit. 

6. Regarding Claim 71 , the applicant argues that Reimer does not disclose object 
properties table and it is not part of the annotation data that is transmitted to the 
receiver as part of "augmented video information" along with "video information" (Page 
17). The applicant argues that Reimer teaches movie source information in a 
foundation information database (Page 17). 
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In response to the argument, Microsoft Press 3'"'^ edition Computer Dictionary 
defines data structure as an organization sclieme, sucli as a record or array, tliat can 
be applied to data to facilitate interpreting the data or performing operations on it. 
Therefore, the data structure of Wistendahl is an organization scheme that can be 
applied to data to facilitate interpreting data or performing operations on it. Reimer 
discloses on the corresponding information data structure is an object properties table 
(Figure 8, 802, Figures 10A-10C, Figure 11, 1106). The object properties table has 
entries (Figures 1 0A-IOC, Column 18, lines 45-57) which includes an information 
category and an identifier to a second data structure providing details for the information 
category (Column 18, lines 45-56, Figure 10A, 1002, Figure 10B, 1012, Figure IOC, 
1018, Figure 8, 802, Column 17, lines 40-47). Wistendahl disclose a corresponding 
data structure (Figure 2, Figure 3). 

Furthermore, in KSR International Co. Teleflex Inc., 127 S.Ct 1727, No. 04-1350, 
slip. op. at 12 (2007), the Court found that if all the claimed elements are known in the 
prior art then one skilled in the art could have combined the elements as claimed by 
known methods with no change in their respective functions, and the combination would 
have yield predictable results to one of ordinary skill in the art at the time of the 
invention. 

7. Regarding the obvious type provisional double patent rejection, the applicant 
states they will submit a Terminal Disclaimer when there is an indication of allowance 
(Page 18). 



Application/Control Number: 09/943,583 
Art Unit: 2623 



Page 7 



In response to the statement, the Examiner notes that the applicant will submit 
the terminal disclaimer. Nevertheless, the double patenting rejection will remain until 
the rejection no longer applies. 

Claim Rejections - 35 USC § 103 

8. The following is a quotation of 35 U.S.C. 1 03(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or deschbed as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the phor art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

9. Claims 1-7, 17-21, 35-41, 50-52, 56 and 63-70 under 35 U.S.C. 103(a) as being 
unpatentable over Srinivasan et al (US 6,357,042 and hereafter referred to as 
"Srinivasan") in view of Wistendahl et al (US 6,496,981 and hereafter referred to as 
"Wistendahl") 

Regarding Claims 1 and 35, Srinivasan discloses a hyperlinked broadcast 
system and a method of generating a hyperlinked video signal (Figure 1, Figure 14, 
Figure 18) comprising: a video headend or video source provides video information for a 
video program including a plurality of consecutive video frames (Figure 1,12, Column 7, 
lines 22-25, Column 8, lines 13-14, 38-49), 

an annotation system tracing or tracking movement of two or more video objects 
appearing in each of plurality of consecutive video frames from a first location to a 
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second location (Column 8, lines 5-37) and generating annotation data (Column 6, lines 
8-19) and annotation data timing information (Column 7, lines 21-30, Column 8, lines 5- 
50), the annotation data including only one mask for each video frame of the plurality of 
consecutive frames (Column 5, lines 54-65, Figure 3, Figure 4), each mask being 
stamped with the corresponding annotation data timing information based on a frame 
time of the corresponding video frame (Abstract, Column 7, lines 21-30, Column 8, lines 
5-50), each mask including location (Figure 4, Column 8, lines 5-50) and graphic data of 
two or more graphic images to be overlaid on two or more video objects or entities in 
the corresponding video frame (Column 5, line 54-65, Column 6, lines 8-19, Column 7, 
lines 33-47); and 

an augmented video information transmission generator receiving the annotation 
data, the video information, and the annotation timing information, the augmented video 
information transmission generator generating an augmented video transmission signal 
comprising annotation data, the annotation data timing information, and the video 
information, transmits the augmented video transmission signal to a receiver (Column 5, 
lines 54-65, Column 6, lines 29-33, Column 11, lines 36-37, Figure 18, Column 3, lines 
53-55, Column 4, lines 3-7, 17-24), 

wherein the augmented video information transmission generator associates the 
video information with the annotation data using the annotation data timing information 
(Column 5, lines 54-65, Column 6, lines 29-33, Column 11, lines 36-37), 
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wherein tlie receiver receiving tlie augmented video transmission signal is 
program for eacli of tlie plurality of consecutive video frames (Column 36, lines 41-49, 
Column 21 , lines 34-52) to: 

compare a current annotation data timing information with a current frame time of 
a current video frame (Abstract, Column 4, lines 35-37, 55-62, Column 5, line 54-65, 
Column 6, lines 8-19, Column 7, lines 33-47, Column 12, lines 21-46, Column 36, lines 
41-49, Column 21, lines 34-52); 

identify the corresponding mask if the comparison results in a match (Column 7, 
lines 36-45, Column 12, lines 21-46, Column 21, lines 34-52); 

retrieve the location and graphics data of the two or more graphics images from 
the identified mask (Abstract, Column 4, lines 35-37, 55-62, Column 5, line 54-65, 
Column 6, lines 8-19, Column 7, lines 33-47, Column 12, lines 21-46, Column 21, lines 
34-52); and 

overlay the two or more graphics images on the corresponding two or more video 
object appearing in the video frame based on the retrieved location and graphics data, 
wherein the overlaying of the two or more graphics images is synchronized on a frame 
by frame basis with the movement of two or more video objects from the first location to 
the second location over the plurality of consecutive video frames (Abstract, Column 4, 
lines 35-37, 55-62, Column 5, line 54-65, Column 6, lines 8-19, Column 7, lines 21-47, 
Column 12, lines 21-46, Column 36, lines 41-49, Column 21, lines 34-52). Srinivasan 
discloses annotation data comprises a plurality of object data packets (Column 21, lines 
14-25). 
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Srinivasan is silent on tlie nnasl< including an identifier to an object mapping table 
including one or more of the plurality of object data packets, the object mapping table 
including at least one entry with an indicia from the corresponding mask identifying a 
particular video object, the entry further storing an identifier to a corresponding one of a 
plurality information data structures included in one or more of the plurality object data 
packets, the information data structures including information for the particular video 
object. Wistendahl discloses on annotation data further includes a plurality of object 
data packets (Figure 3, 32, Column 11, lines 30-65, Column 17, lines 8-10), the mask 
further including an identifier or hotspot to an object mapping table or N data 
information (including values for several objects such as object A, A'(F), and object B, 
B'(F)) included in the object data packets (Column 6, lines 17-38, Column 7, lines 1-10, 
Figure 2, Figure 3, Column 11, lines 30-65, Column 17, lines 8-10). Microsoft Press 3'^^ 
edition Computer Dictionary defines data structure as an organization scheme, such as 
a record or array, that can be applied to data to facilitate interpreting the data or 
performing operations on it. Wistendahl discloses the object mapping table including at 
least one entry with an indicia from the corresponding mask identifying a particular 
video object (Figure 2, Column 6, lines 17-38), the entry further storing an identifier to a 
corresponding one of a plurality of information data structures included in one or more 
of the plurality of object data packets (Figure 2, Figure 3, Column 6, lines 17-38, 
Column 10, lines 8-67, Column 17, lines 1-13, Column 11, lines 30-65,), the information 
data structures including information for particular video object (Column 9, lines 59-67, 
Column 10, lines 1-67, Column 11, lines 1-40, Column 12, lines 1-18, 40-49). 
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Therefore, it would liave been obvious at tlie time tlie invention was made to one 
of ordinary sl<ill in tlie art to modify Srinivasan to include the mask further including an 
identifier or hotspot to an object mapping table or N data information included in the 
object data packets (Column 6, lines 17-38, Column 7, lines 1-10, Figure 2, Figure 3, 
Column 1 1 , lines 30-65, Column 17, lines 8-10), the object mapping table including at 
least one entry with an indicia from the corresponding mask identifying a particular 
video object (Figure 2, Column 6, lines 17-38), the entry further storing an identifier to a 
corresponding one of a plurality of information data structures included in one or more 
of the plurality of object data packets (Figure 2, Figure 3, Column 6, lines 17-38, 
Column 10, lines 8-67, Column 17, lines 1-13, Column 11, lines 30-65,), the information 
data structures including information for particular video object (Column 9, lines 59-67, 
Column 1 0, lines 1 -67, Column 1 1 , lines 1 -40, Column 1 2, lines 1 -1 8, 40-49) as taught 
by Wistendahl in order to use media content for interactive television (Column 1 , lines 
12-14, 45-67) as disclosed by Wistendahl. 

Regarding Claims 63 and 67, Srinivasan discloses a hyperlinked broadcast 
system and a method of generating a hyperlinked video signal comprising: 

a video source providing video information for a video program including a 
plurality of video frames (Figure 1,12, Column 7, lines 22-25, Column 8, lines 13-14, 
38-49); 

receiving video information for a video program including a plurality of video 
frames (Figure 1,12, Column 7, lines 22-25, Column 8, lines 13-14, 38-49) 
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an annotation system generating annotation data and annotation data timing 
information (Column 8, lines 5-37), the annotation data including a plurality of masks 
and a plurality of object data packets (Figure 4, Figure 3, Column 8, lines 5-37), each 
mask corresponding to a particular video frame of a video program (Figure 3, 31 , 33, 
29, 27) and including graphics data associated with one or more video objects in the 
particular video frame (Column 5, line 54-65, Column 6, lines 8-19, Column 7, lines 33- 
47), each mask being associated with the corresponding annotation data timing 
information (Figure 4, Figure 3, Column 5, line 54-65, Column 6, lines 8-19, Column 7, 
lines 33-47), and an object mapping table included in at least a particular one of the 
plurality of object data packets (Figure 4), the object mapping table including an entry 
associated with each of the one or more video objects in the particular video frame 
(Figure 3, Figure 4), communicating the annotation data timing information, the 
annotation data and the video information to an augmented transmission generator 
(Figure 1 , Figure 3, Figure 4) and 

an augmented video information transmission generator receiving the annotation 
data, the video information, and the annotation data timing information, the augmented 
video information transmission generator generating an augmented video transmission 
signal comprising the annotation data, the annotation data timing information, and the 
video information, and transmitting the augmented video transmission signal to a 
receiver, wherein the augmented video information transmission generator associates 
the video information with the annotation data using the annotation data timing 
information (Column 5, lines 54-65, Column 6, lines 29-33, Column 11, lines 36-37, 
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Figure 18, Column 3, lines 53-55, Column 4, lines 3-7, 17-24). Srinivasan discloses 
annotation data comprises a plurality of object data packets (Column 21, lines 14-25). 
Srinivasan is silent on object mapping table as claimed. 

Wistendahl discloses on annotation data further includes a plurality of object data 
packets (Figure 3, 32, Column 11, lines 30-65, Column 17, lines 8-10), the mask further 
including an identifier or a hyperlink to an object mapping table or N data information 
(including values for several objects such as object A, A'(F), and object B, B'(F))) 
(Column 6, lines 17-38, Column 7, lines 1-10). Microsoft Press 3'"'^ edition Computer 
Dictionary defines data structure as an organization scheme, such as a record or array, 
that can be applied to data to facilitate interpreting the data or performing operations on 
it. Wistendahl discloses the object mapping table including at least one entry with an 
indicia from the corresponding mask identifying a particular video object or one entry 
associated with each of the one or more video objects in the particular video frame 
(Figure 2, Column 6, lines 17-38), each entry in the object mapping table storing an 
identifier to one or one or more information data structures included in one or more of 
the plurality object data packets (Figure 2, Figure 3, Column 6, lines 17-38, Column 10, 
lines 8-67, Column 17, lines 1-13, Column 11, lines 30-65), the information data 
structures including information for particular video object (Column 9, lines 59-67, 
Column 10, lines 1-67, Column 11, lines 1-40, Column 12, lines 1-18, 40-49). 

Therefore, it would have been obvious at the time the invention was made to one 
of ordinary skill in the art to modify Srinivasan to include the mask further including an 
identifier to an object mapping table or N data information (including values for several 
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objects) (Column 6, lines 17-38, Column 7, lines 1-10) such as including at least one 
entry with an indicia from the corresponding mask identifying a particular video object 
(Figure 2, Column 6, lines 1 7-38), the entry storing an identifier to one or one or more 
information data structures included in one or more of the plurality object data packets 
(Figure 2, Figure 3, Column 6, lines 17-38, Column 10, lines 8-67, Column 17, lines 1- 

13, Column 11, lines 30-65), the information data structures including information for 
particular video object (Column 9, lines 59-67, Column 10, lines 1-56) as taught by 
Wistendahl in order to use media content for interactive television (Column 1 , lines 12- 

14, 45-67), to author exiting media content using tools to reduce development time 
(Column 2, lines 43-46) and so that media contact is kept uncorrupted so that the 
content can be displayed on any media delivery system or display platform (Column 3, 
lines 2-5) as disclosed by Wistendahl. 

Regarding Claims 2 and 36, Srinivasan and Wistendahl disclose all the 
limitations of Claims 1 and 35 respectively. Srinivasan discloses the augmented video 
information transmission generator comprises a component, which inserts data into the 
VBI or a vertical blanking insertion device (Column 35, lines 60-62, Column 36, lines 36- 
40). 

Regarding Claims 3 and 37, Srinivasan and Wistendahl disclose all the 
limitations of Claims 1 and 35 respectively. Srinivasan discloses that the augmented 
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video information transmission generator comprises a digital video data multiplexer 
(Column 36, lines 25-32). 

Regarding Claims 4 and 38, Srinivasan and Wistendahl disclose all the 
limitations of Claims 1 and 35 respectively. Srinivasan discloses that he timing 
information comprises at least one of timestamp information and a frame number 
information (Column 3, Column 4, lines 3-16, Column 7, line 23, Column 8, lines 5-14). 

Regarding Claims 5 and 39, Srinivasan and Wistendahl disclose all the 
limitations of Claims 1 and 35 respectively. Srinivasan discloses the programs 
comprise digital video signal (Column 36, lines 25-27). 

Regarding Claims 6 and 40, Srinivasan and Wistendahl disclose all the 
limitations of Claims 1 and 35 respectively. Srinivasan discloses the programs 
comprise analog video signal (Column 36, lines 36-40). 

Regarding Claims 7, Srinivasan and Wistendahl disclose all the limitations of 
Claim 1. Srinivasan disclose a post production environment (Figure 1,11, Figure 18, 
251, 253, 255), and a headend comprising the augmented video information 
transmission generator (Column 6, lines 33-35, Figure 18, 259), the video information 
and annotation data timing information are combined by the post production 
environment and transmitted to the headend (Column 7, lines 20-48, Column 8, lines 5- 
49, Column 35, lines 20-36). 

Regarding Claims 8, Srinivasan and Wistendahl disclose all the limitations of 
Claim 7. Wistendahl discloses that the headend is a cable headend (Column 16, lines 
6-8). 
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Regarding Claim 14, Srinivasan and Wistendalil disclose all the limitations of 
Claims 1 and 35 respectively. Srinivasan discloses displaying annotation data in 
response to a viewer request (Column 6, lines 8-19, Column 12, lines 21-40). 

Regarding Claims 17 and 46, Srinivasan and Wistendahl disclose all the 
limitations of Claims 1 and 35 respectively. Srinivasan discloses that the mask 
comprises location information of two or more objects or entities in an annotated video 
frame in the corresponding video frame (Column 5, lines 54-65, Column 7, lines 22-48, 
Column 8, lines 5-50). 

Regarding Claims 18 and 47, Srinivasan and Wistendahl disclose all the 
limitations of Claims 17 and 46 respectively. Srinivasan discloses the location 
information includes a graphics location reference that represents a fixed relation to a 
set of pixels associated with each object (Column 9, lines 19-55, Figure 4). 

Regarding Claims 19 and 48, Srinivasan and Wistendahl disclose all the 
limitations of Claims 1 8 and 47 respectively. Srinivasan discloses a table of pixels 
based on a tracking box and object (Figure 4). Therefore, it is necessarily included that 
the pixels include the upper left most pixel in the associated pixel set if the object that is 
being tracked is located at the upper most left corner (Figure 4, Figure 3). 

Regarding Claims 20 and 49, Srinivasan and Wistendahl disclose all the 
limitations of claims 1 8 and 48 respectively. Srinivasan discloses a table of pixels 
based on a tracking box and object (Figure 4), if the object is in the center position then 
the centroid pixel is in the associated pixel set (Figure 4, Figure 3). Therefore, it is 
necessarily included that the pixels include the centroid pixel in the associated pixel set. 
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Regarding Claim 41, Srinivasan and Wistendalil disclose all the limitations of 
Claim 35. Srinivasan discloses inserting the annotation data timing information in a 
vertical blanking interval of an analog video signal (Column 36, lines 36-41). 

Regarding Claim 43, Srinivasan and Wistendahl disclose all the limitations of 
Claim 35. Srinivasan discloses the transmitting the timing information and video 
information to a broadcast network and subsequently to the augmented video 
transmission generator (Column 5, lines 54-65, Column 6, lines 29-40, Column 7, lines 
22-25, Column 36, lines 25-40). 

Regarding Claims 21 and 50, Shoff discloses all the limitations of Claims 1 and 
35 respectively. Srinivasan discloses that the mask comprises location information 
about an object in a video frame to be annotated or supplemental content to be added 
to video (Figure 4). Srinivasan is silent on the shape information. In analogous art, 
Wistendahl discloses that location and shape information of an object in the video frame 
(Column 10, lines 8-35). Therefore, it would have been obvious at the time the 
invention was made to one of ordinary skill in the art to modify Shoff to include shape 
information of an object in the video frame (Column 10, lines 8-35) as taught by 
Wistendahl in order to use media content for interactive television (Column 1 , lines 12- 
14, 45-67) as disclosed by Wistendahl. 

Regarding Claim 51, Srinivasan and Wistendahl disclose all the limitations of 
Claim 50. Wistendahl discloses the shape information is represented by a hyper link 
(Column 10, lines 36-56). The hyperlink can be a graphical overlay of the object 
(Column 9, lines 28-33). 
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Regarding Claim 52, Srinivasan and Wistendalil disclose all the limitations of 
Claim 50. Wistendahl discloses the shape information is represented by an outline of 
the object (Column 10, lines 25-28). 

Regarding Claim 56, Srinivasan and Wistendahl disclose all the limitations of 
Claim 1 . Wistendahl discloses the receiver being configured to overlay a graphics 
image on a particular video frame for the particular video object based on the graphics 
data included in the corresponding mask (Column 9, lines 59-67, Column 10, lines 1- 
56), retrieve the identifier of the object mapping table from the corresponding mask 
response to a user selection associated with the overlaid graphics image (Column 6, 
lines 60-67, Column 7, lines 1-10, Column 9, lines 59-67, Column 10, lines 1-56); 
retrieve the object mapping table based on the retrieved identifier (Figure 3, Figure 7a); 
identify the indicia in the corresponding mask for the particular video object for which 
the graphics image was overlaid (Column 15, lines 3-5); locate the entry in the object 
mapping table with the identified indicia (Figure 5b, 51b, Figure 7a, Figure 7B); ); 
retrieving from the located entry the identify of the corresponding information data 
structures identified by the retrieved identifier (Column 6, lines 60-67, Column 7, lines 1- 
10, Column 15, lines 3-5); locate the entry in the object mapping table with the identified 
indicia (Figure 5b, 51b, Figure 7a, Figure 7B); identify the information data structures 
associated with the located entry (Figure 5a, Figure 5b); retrieve the information data 
structures identified by retrieved identifier (Figure 2, Figure 3); and display information 
based on the retrieved information data structure on the display device (Figure 7a, 
Column 13, lines 54-62). 
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Regarding Claims 64 and 68, Srinivasan and Wistendalil disclose all the 
limitations of Claims 63 and 67 respectively. Srinivasan discloses comparing a current 
annotation data timing information with a current frame time of a current video frame 
(Abstract, Column 4, lines 35-37, 55-62, Column 5, line 54-65, Column 6, lines 8-19, 
Column 7, lines 33-47, Column 12, lines 21-46, Column 36, lines 41-49, Column 21, 
lines 34-52); retrieve the location and graphics data of the graphics images from the 
identified mask if the comparison results in a match (Abstract, Column 4, lines 35-37, 
55-62, Column 5, line 54-65, Column 6, lines 8-19, Column 7, lines 33-47, Column 12, 
lines 21-46, Column 36, lines 41-49, Column 21 , lines 34-52); and overlay one or more 
graphics images generated based on the retrieved graphics data on the one or more 
video objects appearing in the current video frame (Abstract, Column 4, lines 35-37, 
55-62, Column 5, line 54-65, Column 6, lines 8-19, Column 7, lines 21-47, Column 12, 
lines 21-46, Column 36, lines 41-49, Column 21, lines 34-52). 

Regarding Claims 65 and 69, Srinivasan and Wistendahl disclose all the 
limitations of Claims 64 and 68 respectively. Srinivasan discloses the receiver is further 
programmed to draw the one or more graphics images on a frame-by-frame basis 
based on the graphics data included in a plurality of masks, the drawing of the one or 
more graphics images being synchronized to the corresponding video frame based on 
the annotation data timing information associated with the plurality of masks (Abstract, 
Column 4, lines 35-37, 55-62, Column 5, line 54-65, Column 6, lines 8-19, Column 7, 
lines 21-47, Column 12, lines 21-46, Column 36, lines 41-49, Column 21, lines 34-52). 
Wistendahl discloses drawing one or more graphic images on a frame-by-frame basis 
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based on the graphics data included in a plurality of masks (Column 1 4, lines 1 0-21 , 
Column 10, lines 8-61). 

Regarding Claims 66 and 70, Srinivasan and Wistendahl disclose all the 
limitations of Claims 65 and 69 respectively. Wistendahl discloses the receiver being 
configured to overlay a graphics image on a particular video frame for the particular 
video object based on the graphics data included in the corresponding mask (Column 9, 
lines 59-67, Column 10, lines 1-56), receive a user selection associated with one of the 
overlaid graphics images for a particular video frame (Column 15, lines 3-5), retrieve the 
identifier of the object mapping table from the corresponding mask response to a user 
selection associated with the overlaid graphics image (Column 6, lines 60-67, Column 
7, lines 1-10, Column 9, lines 59-67, Column 10, lines 1-56); retrieve the object mapping 
table based on the retrieved identifier (Figure 3, Figure 7a); locate the entry in the object 
mapping table for the video object associated with the user selection (Column 6, lines 
60-67, Column 7, lines 1-10, Figure 5b, 51b, Figure 7a, Figure 7B, Column 13, lines 1- 
25, Column 14, lines 10-21); retrieving from the located entry the identify of the 
corresponding information data structures identified by the retrieved identifier (Column 
6, lines 60-67, Column 7, lines 1-10, Column 15, lines 3-5); locate the entry in the object 
mapping table with the identified indicia (Figure 5b, 51b, Figure 7a, Figure 7B); identify 
the information data structures associated with the located entry (Figure 5a, Figure 5b); 
retrieve the information data structures identified by retrieved identifier (Figure 2, Figure 
3); and display information based on the retrieved information data structure on the 
display device (Figure 7a, Column 13, lines 54-62). 
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10. Claim 9 is rejected under 35 U.S.C. 103(a) as being unpatentable over 
Srinivasan in view of Wistendahl, as applied to Claim 7 above, further in view of Shoff et 
al (US 6,240,555 and hereafter referred to as "Shoff"). 

Regarding Claims 9, Srinivasan and Wistendahl disclose all the limitations of 
Claim 7. Srinivasan and Wistendahl do not to disclose the headend is a satellite 
headend. In analogous art, Shoff discloses a post production environment (Figure 2, 
22), a broadcast network (Column 4, lines 43-50), and the post production environment 
combines video data and synchronized timing data and transmits to a headend or node 
(Column 4, lines 43-50). Shoff discloses that the node is a satellite headend (Column 4, 
lines 45-52). Therefore, it would have been obvious at the time the invention was made 
to one of ordinary skill in the art to modify Srinivasan to include that the headend is a 
satellite headend (Column 4, lines 45-52) as taught by Shoff in order to enable viewer 
interactively with video program (Column 1 , lines 8-14) as disclosed by Shoff. 

1 1 . Claim 42 is rejected under 35 U.S.C. 103(a) as being unpatentable over 
Srinivasan in view of Wistendahl, as applied to Claim 35 above, further in view of Oguro 
et al (US 2001/0033739 and hereafter referred to as "Oguro"). 

Regarding Claim 42, Srinivasan and Wistendahl disclose all the limitations of 
Claim 35. Srinivasan disclose inserting timing information into the VBI of an analog 
signal (Column 36, lines (Column 25-40). Srinivasan and Wistendahl are silent the 
insertion of data in the vertical ancillary data or VBI of a digital video signal. Oguro 
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discloses a television broadcast system, which transmits digital video signals to the user 
(Page 5, paragraph 0077) and inserts data into the VBI of a digital video signal (Page 5, 
paragraph 0076). Therefore, it would have been obvious at the time the invention was 
made to one of ordinary skill in the art to modify the combination to include inserting 
information in the VBI of a digital video signal (Page 5, paragraph 0076) as taught by 
Oguro in order to provide copy protect television broadcast programs of a digital video 
signal (Page 1, paragraphs 0001-0006) as disclosed by Oguro. 

12. Claims 61 , 62 and 72 are rejected under 35 U.S.C. 1 03(a) as being unpatentable 
over Srinivasan in view of Wistendahl, as applied to Claims 1 and 35 above, further in 
view of Kaiser et al (US 6,615,408 and hereafter referred to as "Kaiser"). 

Regarding Claims 61 and 62, Srinivasan and Wistendahl disclose all the 
limitations of Claims 1 and 35 respectively. Srinivasan discloses overlaying two or more 
graphics images for alerting a viewer of the interactive data (Column 6, lines 6-19). 
Srinivasan and Wistendahl do not explicitly disclose the overlaying of the graphics 
images is for alerting a viewer of the interactive data associated prior to the viewer 
transmitting an interactive command with respect to one of video object or video 
objects. Kaiser discloses overlaying of the two or more graphics images is for alerting a 
viewer of the interactive data associated with the two or more video objects prior to the 
viewer transmitting an interactive command with respect to one of the two or more video 
objects (Figure 6B, Column 10, lines 20-41). Therefore, it would have been obvious at 
the time the invention was made to one of ordinary skill in the art to modify the 
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combination to include overlaying of the two or more graphics images is for alerting a 
viewer of the interactive data associated with the two or more video objects prior to the 
viewer transmitting an interactive command with respect to one of the two or more video 
objects (Figure 6B, Column 10, lines 20-41) as taught by Kaiser in order display a 
locator or locators for a video object or objects without clutter video (Column 1 , lines 8- 
1 1 , 34-40) as disclosed by Kaiser and to make it easier for a user so that they do not 
have to determine what is interactive and provides more information. 

Regarding Claim 72, Srinivasan and Wistendahl disclose all the limitations of 
Claim 1 . Srinivasan discloses overlaying two or more graphics images for alerting a 
viewer of the interactive data in which a mask includes data enabled to identify video 
objects in a particular video shot (Column 6, lines 6-19). Srinivasan and Wistendahl do 
not disclose a visibility bit indicative of visually identify an object. Kaiser discloses a 
mask or hotspot includes a visibility bit or highlight information indicative of whether 
video objects appearing in the corresponding video frame are enabled for being visually 
identified for a particular video shot (Column 10, lines 20-42). Therefore, it would have 
been obvious at the time the invention was made to one of ordinary skill in the art to 
modify the combination to include a visibility bit indicative of whether video objects 
appearing in the corresponding video frame are enabled for being visually identified for 
a particular video shot (Column 10, lines 20-42) as taught by Kaiser in order display a 
locator or locators for a video object or objects without clutter video (Column 1 , lines 8- 
1 1 , 34-40) as disclosed by Kaiser and to make it easier for a user so that they do not 
have to determine what is interactive and provides more information. 
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13. Claim 71 is rejected under 35 U.S.C. 103(a) as being unpatentable over 
Srinivasan in view of Wistendahl, as applied to Claim 1 above, further in view of Reimer 
et al (US 5,553,221 and hereafter referred to as "Reimer"). 

Regarding Claim 71, Srinivasan and Wistendahl disclose all the limitations of 
Claim 1. Wistendahl disclose a corresponding data structure (Figure 2, Figure 3). 
Srinivasan and Wistendahl are silent on the corresponding information data structure is 
an object properties table storing one or more second entries, wherein each of the one 
or more second entries includes an information category and an identifier to a second 
one of the plurality of information data structures providing details for the information 
category. Microsoft Press 3'"'^ edition Computer Dictionary defines data structure as an 
organization scheme, such as a record or array, that can be applied to data to facilitate 
interpreting the data or performing operations on it. In analogous art, Reimer discloses 
on the corresponding information data structure is an object properties table (Figure 8, 
802, Figures 1 0A-IOC, Figure 11, 1106) storing one or more second entries (Figures 
1 0A-IOC, Column 18, lines 45-57), wherein each of the one or more second entries 
includes an information category and an identifier to a second one of the plurality of 
information data structures providing details for the information category (Column 18, 
lines 45-56, Figure 10A, 1002, Figure 10B, 1012, Figure IOC, 1018, Figure 8, 802, 
Column 17, lines 40-47). Therefore, it would have been obvious at the time the 
invention was made to one of ordinary skill in the art to modify the combination to 
include corresponding information data structure is an object properties table (Figure 8, 
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802, Figures 10A-10C, Figure 11, 1106) storing one or more second entries (Figures 
1 0A-IOC, Column 18, lines 45-57), wherein each of the one or more second entries 
includes an information category and an identifier to a second one of the plurality of 
information data structures providing details for the information category (Column 18, 
lines 45-56, Figure 10A, 1002, Figure 10B, 1012, Figure IOC, 1018, Figure 8, 802, 
Column 17, lines 40-47) as taught by Reimer in order to link and present information of 
movies with underlying source information (Column 1 , lines 8-1 1 , 34-40) as disclosed by 
Reimer. 



Double Patenting 



14. The nonstatutory double patenting rejection is based on a judicially created 
doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the 
unjustified or improper timewise extension of the "right to exclude" granted by a patent 
and to prevent possible harassment by multiple assignees. A nonstatutory 
obviousness-type double patenting rejection is appropriate where the conflicting claims 
are not identical, but at least one examined application claim is not patentably distinct 
from the reference claim(s) because the examined application claim is either anticipated 
by, or would have been obvious over, the reference claim(s). See, e.g.. In re Berg, 140 
F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 
USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 
1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, All 
F.2d 438, 164 USPQ 619 (CCPA 1970); and In re Thorington, 418 F.2d 528, 163 
USPQ 644 (CCPA 1969). 

A timely filed terminal disclaimer in compliance with 37 CFR 1 .321 (c) or 1 .321 (d) 
may be used to overcome an actual or provisional rejection based on a nonstatutory 
double patenting ground provided the conflicting application or patent either is shown to 
be commonly owned with this application, or claims an invention made as a result of 
activities undertaken within the scope of a joint research agreement. 

Effective January 1 , 1994, a registered attorney or agent of record may sign a 
terminal disclaimer. A terminal disclaimer signed by the assignee must fully comply with 
37 CFR 3.73(b). 
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15. Claim 1 is rejected on tlie ground of nonstatutory obviousness-type double 
patenting as being unpatentable over claim 1 of U.S. Patent No. 7,249,367 in view of 
Srinivasan. 

The instant application's limitations of" a hyperlinked broadcast system 
comprising: a video source providing video program including a plurality of consecutive 
video frames" is met by the limitations of "A hyperlinked video broadcast system" of U.S. 
Patent No. 7,249,367. 

The instant application's limitations of "an annotation system generating 
annotation data, the annotation data including only one mask for each video frame of 
the plurality of consecutive frames, each mask being stamped with the corresponding 
annotation data timing information based on a frame time of the corresponding video 
frame, each mask including location and graphic data of two or more graphic images to 
be overlaid on two or more video objects or entities in the corresponding video frame; 
the annotation data further including a plurality of object data packets, mask including 
an identifier to an object mapping table including one or more of the plurality of object 
data packets, the object mapping table including at least one entry with an indicia from 
the corresponding mask identifying a particular video object, the entry further storing an 
identifier to a corresponding one of a plurality information data structures included in 
one or more of the plurality object data packets, the information data structures 
including information for the particular video object " is met by the limitations " 
a mask generator generating a plurality of masks, each mask corresponding to a 
particular video frame of a video program, each mask including graphics data 
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associated witli one or more video objects in tlie particular video frame, eacli masl< 
including an identifier to an object mapping table including an entry associated with 
each of the one or more video objects in the particular video frame, each entry in the 
object mapping table referencing one or more information data structures including 
information associated with the corresponding video object; an annotation source 
providing a plurality of object data packets including the object mapping table and the 
one or more information data structures for the one or more video objects" of U.S. 
Patent No. 7,249,367. 

The instant application's limitations of an augmented video information 
transmission generator receiving the annotation data, the video information, the 
augmented video information transmission generator generating an augmented video 
transmission signal comprising annotation data, the and the video information, transmits 
the augmented video transmission signal to a receiver" is met by limitation "a transmitter 
transmitting the television broadcast signal" of U.S. Patent No. 7,249,367. 

wherein the augmented video information transmission generator associates the 
video information with the annotation data using the annotation data timing information, 
wherein the receiver receiving the augmented video transmission signal is program for 
each of the plurality of consecutive video frames to; identify the corresponding mask if 
the comparison results in a match; retrieve the location and graphics data of the two or 
more graphics images from the identified mask; and overlay the two or more graphics 
images on the corresponding two or more video object appearing in the video frame 
based on the retrieved location and graphics data, wherein the overlaying of the two or 
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more graphics images is synclironized on a frame by frame basis witli tlie movement of 
two or more video objects from tlie first location to tlie second location over the plurality 
of consecutive video frames" is met by limitations "wherein the receiver is programmed 
to retrieve and overlay the graphics image corresponding to the particular video object 
on the corresponding video frame, receive viewer actuation of the overlaid graphics 
image, and in response to the viewer actuation, review the indicia in the particular 
information data structure corresponding to the particular video object for determining 
whether the particular video object is linked to one of the plurality of multiplexed 
program streams, and in response to a determination that the particular video object is 
linked to one of the plurality of multiplexed program streams, retrieve from the data 
particular information data structure the identifier of the particular one of the plurality of 
multiplexed program streams, and switch from presenting a current multiplexed program 
stream to presenting the particular one of the plurality of multiplexed program streams" 
of U.S. Patent No. 7,249,367. 

The instant application is missing including "a plurality of multiplexed program 
streams, an identifier for a particular one of the plurality of multiplexed program streams, 
a particular one of the information data structures including an indicia indicative that a 
particular one of the one or more video objects is linked to one of the plurality of 
multiplexed program streams, and an encoder encoding the plurality of masks and the 
plurality of object data packets into a television broadcast signal; a transmitter 
transmitting the plurality of multiplexed program streams to a receiver." It would be 
obvious to modify U.S. Patent No. 7,249,367 to make the claim broader. 
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The instant application's "an annotation system tracl<ing movement of two or 
more video objects appearing in eacli of plurality of consecutive video frames from a 
first location to a second location and generating annotation data timing information, an 
augmented video information transmission generator generating the annotation timing 
information, the augmented video information transmission generator generating an 
augmented video transmission signal comprising annotation data, the annotation data 
timing information, and the video information, transmits the augmented video 
transmission signal to a receiver, wherein the receiver compare a current annotation 
data timing information with a current frame time of a current video frame" are additional 
features. 

It would be obvious to include the limitations in U.S. Patent 7,249,367 as they are 
disclosed by prior art. Srinivasan discloses an annotation system generating annotation 
data timing information (Column 7, lines 21-30, Column 8, lines 5-50), the augmented 
video information transmission generator receiving the annotation timing information, 
the augmented video information transmission generator generating an augmented 
video transmission signal comprising annotation data, the annotation data timing 
information, and the video information(Column 5, lines 54-65, Column 6, lines 29-33, 
Column 11, lines 36-37, Figure 18, Column 3, lines 53-55, Column 4, lines 3-7, 17- 
24),wherein the augmented video information transmission generator associates the 
video information with the annotation data using the annotation data timing information 
(Column 5, lines 54-65, Column 6, lines 29-33, Column 11, lines 36-37),wherein the 
receiver compare a current annotation data timing information with a current frame time 
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of a current video frame (Abstract, Column 4, lines 35-37, 55-62, Column 5, line 54-65, 
Column 6, lines 8-19, Column 7, lines 33-47, Column 12, lines 21-46, Column 36, lines 
41-49, Column 21, lines 34-52). 

Conclusion 

16. The prior art made of record and not relied upon is considered pertinent to 
applicant's disclosure. Hopcroft et al (US 6,359,629) discloses a visibility bit (Abstract, 
Column 8, lines 9-18). 

17. THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time 
policy as set forth in 37 CFR 1 .136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1 .136(a) will be calculated from the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the mailing date of this final action. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to FARZANA E. HOSSAIN whose telephone number is 
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(571)272-5943. The examiner can normally be reached on Monday to Friday 7:30 am 
to 3:00 pm. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Christopher Kelley can be reached on 571-272-7331 . The fax phone 
number for the organization where this application or proceeding is assigned is 571- 
273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 

/Chris Kelley/ 

Supervisory Patent Examiner, Art 
Unit 2623 

FEH 

June 10, 2008 



